Method, apparatus, and device for market dynamic reporting strategy based on reinforcement learning

Abstract

The application relates to the fields of artificial intelligence and power grid control, in particular to a method, device and equipment for market dynamic declaration strategy based on reinforcement learning, which comprises the following steps: acquiring basic operation data of a comprehensive energy system; obtaining optimal matching analysis between strategies and scenes based on the basic operation data of the comprehensive energy system; and obtaining a declaration scheme by screening optimal strategies in a declaration strategy library based on the optimal matching analysis. The application helps to realize adaptive decision of declaration strategies by mining the internal matching relationship between declaration strategies and decision scenes, select the most suitable declaration strategy method according to the characteristics of actual decision scenes, formulate a declaration scheme, and further effectively improve the accuracy of spot market declaration decision of the comprehensive energy system and the market transaction benefit.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and power grid control technology, and in particular to methods, apparatus and devices for market dynamic reporting strategies based on reinforcement learning. Background Technology

[0002] As my country's power market reform deepens and policies to liberalize power generation and consumption plans are implemented more rapidly, the impact of power market transactions on the operational efficiency of power generation and consumption enterprises is becoming increasingly prominent, giving rise to the issue of market participants' transaction reporting strategies. In particular, with the accelerated development of the spot market, the complexity of market transactions has significantly increased, making the need for efficient and accurate market transaction reporting strategies increasingly urgent for market participants. Existing technologies have certain limitations in reporting strategies; they cannot be fully applied to the market environment during the actual bidding decision-making process of the integrated energy system, and cannot meet the self-adaptation requirements of reporting strategies and the bidding environment, thus preventing market transaction efficiency from reaching the expected level. Summary of the Invention

[0003] To overcome, to at least some extent, the problem that relevant technology application strategies cannot be fully applied to the market environment during the actual bidding decision-making period of integrated energy systems, and cannot meet the adaptation requirements of application strategies and bidding environment, this application provides a method, apparatus, and equipment for a market dynamic application strategy based on reinforcement learning.

[0004] The proposed solution is as follows: On the one hand, this application provides a method for market dynamic reporting strategies based on reinforcement learning, the method comprising: Obtain basic operational data for integrated energy systems; Based on the basic operational data of the integrated energy system, a smart agent model for the application strategy is constructed to obtain the optimal matching analysis between the strategy and the scenario; Based on the optimal matching analysis, the optimal strategy is selected from the application strategy library to obtain the application plan.

[0005] Furthermore, the intelligent agent model for the declaration strategy is constructed, including: By using the Q-reinforcement learning algorithm, we designed its three elements: environmental variables, action space, and reward function, and obtained the optimal matching analysis between policy scenarios. The environmental variables include: market transaction variables, including: benchmark power generation demand, new energy power generation forecast, conventional unit operating capacity, and fuel price index; The category of self-characteristics includes: self-electricity demand, self-heating demand, self-cooling demand, self-new energy power generation forecast, and self-energy storage capacity.

[0006] Furthermore, the action space includes: A set of application strategies for integrated energy systems is obtained through agent-based decision-making. Based on the obtained set of integrated energy system declaration strategies, the optimal declaration strategy under the trading scenario described by the environmental variables is obtained. The formula for obtaining the optimal reporting strategy under the trading scenario described by the environmental variables is:

[0007] In the formula, The number of selectable declaration strategies in the declaration strategy library within the action space is the integrated energy system declaration strategy set. , ... These are action status variables that sequentially indicate whether the 1st, 2nd, to Nth reporting strategies have been selected, with values ​​of 0 or 1. When the value is 1, it indicates that the reporting strategy is selected; Otherwise, it indicates that the strategy was not selected.

[0008] Furthermore, the step of obtaining an application scheme by selecting the optimal strategy from the application strategy library based on the optimal matching analysis includes: Based on the basic operational data of the integrated energy system, the application scheme is obtained by training the constructed intelligent agent model of the application strategy using a greedy strategy.

[0009] Secondly, this application provides an apparatus for a market dynamic reporting strategy based on reinforcement learning, the apparatus comprising: The acquisition module is used to acquire basic operational data of the integrated energy system; The strategy construction module is used to obtain the optimal matching analysis between strategies and scenarios based on the basic operational data of the integrated energy system. The strategy training module is used to obtain the application scheme by selecting the optimal strategy from the application strategy library based on the optimal matching analysis.

[0010] Thirdly, this application provides an electronic device, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described above.

[0011] The technical solution provided in this application may include the following beneficial effects: This application obtains basic operational data of the integrated energy system; based on this data, it performs an optimal matching analysis between strategies and scenarios; and based on this analysis, it selects the optimal strategy from a strategy database to obtain a submission plan. This application helps to achieve adaptive decision-making in submission strategies by exploring the inherent matching relationship between submission strategies and decision-making scenarios. It selects the most suitable submission strategy method and formulates a submission plan based on the characteristics of the actual decision-making scenario, further effectively improving the accuracy of submission decisions in the integrated energy system spot market and enhancing its market trading efficiency.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0014] Figure 1 This is a flowchart of a method for a market dynamic reporting strategy based on reinforcement learning, provided in one embodiment of this application. Figure 2 This is a structural diagram of a market dynamic reporting strategy based on reinforcement learning provided in another embodiment of this application; Figure 3 This is a schematic diagram illustrating the changing trend of the reward function during the training process, provided in another embodiment of this application. Figure 4 This is a schematic diagram of the decision ideality test results provided in another embodiment of this application; Figure 5 This is a schematic diagram illustrating the application scenarios of different filing strategies provided in another embodiment of this application; Figure 6 This is a schematic diagram of an electronic device provided in another embodiment of this application. Detailed Implementation

[0015] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0016] Example 1 Figure 1 This is a flowchart illustrating a market dynamic reporting strategy based on reinforcement learning, provided in one embodiment of this application. Please refer to... Figure 1This application provides a method for market dynamic reporting strategies based on reinforcement learning, including: S1. Obtain basic operational data for the integrated energy system; S2. Based on the basic operational data of the integrated energy system, an optimal matching analysis between strategies and scenarios is obtained; S3. Based on the optimal matching analysis, the optimal strategy is selected from the application strategy library to obtain the application plan.

[0017] For step S2, in one embodiment, obtaining the optimal matching analysis between strategies and scenarios based on the basic operational data of the integrated energy system includes: A declaration strategy intelligent agent model is constructed, and its environmental variables, action space, and reward function are designed using the Q reinforcement learning algorithm to obtain the optimal matching analysis between strategy scenarios.

[0018] In this embodiment, reinforcement learning is a machine learning algorithm based on the interaction between an agent and its external environment to update and improve the agent's policy, thereby obtaining an agent that meets the requirements for policy formulation. Reinforcement learning can be represented by a quadruple as follows: (1) In the formula, E represents the reinforcement learning algorithm. These represent the set of environment variables, the set of action space, the reward function, and the action policy, respectively.

[0019] In this embodiment, Q-reinforcement learning will be used as the core algorithm for the application strategy agent. Q-reinforcement learning is essentially a decision-making algorithm based on Markov processes. Its significant advantage lies in its model-free nature; that is, it does not require detailed modeling of the external environment and the decision-making agent. It only requires defining the environmental variables, action space, and reward function of the problem under study, and then implementing training and testing according to the algorithm requirements. Once the requirements are met, it can be applied. As the most widely researched and applied machine learning algorithm currently, Q-reinforcement learning has been comprehensively studied and applied in many fields such as power system dispatching and operation, and market trading.

[0020] In this embodiment of the application, the purpose of constructing the declaration strategy intelligent agent model is to design its environmental variables, action space, and reward function according to the requirements of reinforcement learning application scenarios.

[0021] In one embodiment, the environment variables include the following: Environmental variables, or the basis for an agent's decision-making, are essentially the extraction of characteristics of the agent's decision-making problem. In the real-time market declaration decision-making problem of the integrated energy system studied in this application, both the game-playing factors among market participants and the operating status of the integrated energy system itself need to be considered. Theoretically, the more environmental variables selected during the agent's construction, the richer the information available for the agent's decision-making. However, due to the limitations of reinforcement learning implementation efficiency and considering the potential interference from an increased number of environmental variables, it is generally required that the number of environmental variable indicators for a Q-reinforcement learning agent does not exceed 10. Based on a review of the characteristics of integrated energy system market transaction declaration decision-making models in current research, this application selects 9 indicators as environmental variables. According to the differences in the types of information contained in these indicators, they can be divided into two categories: market transaction and self-characteristics.

[0022] (1) Market-traded items include four items: target power generation demand, new energy power generation forecast, conventional unit operating capacity, and fuel price index. Among them, power generation demand, new energy power generation forecast, and conventional unit operating capacity are all derived from publicly disclosed information on the power trading platform. The above three aspects of information actually represent the tightness of the supply and demand relationship of the entire network during the trading period; while the fuel price index is derived from publicly disclosed information by the local energy authorities and is used to quantify the variable cost price level of conventional power sources.

[0023] (2) Self-characteristic category, including five items: self-electricity demand, self-heating demand, self-cooling demand, self-new energy power generation forecast, and self-energy storage capacity. All of the above indicators are derived from the integrated energy system's own operation and control system. Electricity demand, heating demand, and cooling demand summarize the demand for different types of energy during the trading period of the integrated energy system, while new energy power generation forecast and energy storage capacity reflect its own power supply and regulation capabilities.

[0024] In fact, the aforementioned two environmental variables correspond to key information used in market game analysis and self-operation optimization under the two technical paths of pricing participants and price takers, respectively. In a fully competitive market, when using a marginal cost pricing model, market participants tend to bid based on their own operating costs. In this case, the market game outcome is determined by market supply and demand. New energy sources such as wind power and photovoltaics are low-priced power sources and generally exit the market first. Therefore, the sufficiency of market supply and demand is mainly determined by power generation demand, new energy power generation forecasts, and the operating capacity of conventional power sources. Furthermore, in a comprehensive energy system, electricity, heat, cooling, and other types of energy have complementary characteristics. Therefore, in its optimization decisions, the demand for different types of energy and its own supply capacity must be considered holistically.

[0025] Specifically, the above-mentioned index data are all continuous values. In order to meet the requirements of the Q reinforcement learning algorithm, data processing methods such as range bounding and segment bounding need to be used to convert them into discrete indices.

[0026] In this embodiment of the application, the action space includes: The action space refers to the set of possible actions after an agent makes a decision. In the integrated energy system application strategy problem studied in this application, the agent's decision result is the integrated energy system application strategy. Unlike other application strategies, the dynamic application strategy proposed in this application does not directly output the integrated energy system market application plan, but rather selects the optimal application strategy under the trading scenario described by the environmental variable from the application strategy library, which can be expressed as:

[0027] In the formula, N represents the number of selectable declaration strategies in the declaration strategy library within the action space. , This is an action status variable that sequentially indicates whether the 1st, 2nd, to Nth reporting strategy is selected, with a value of 0 or 1. A value of 1 indicates that the reporting strategy is selected; otherwise, it indicates that the strategy is not selected.

[0028] In this embodiment, the reward function serves as the basis for improving the agent's strategy; essentially, it is an evaluation function for the execution effect of the action strategy formulated during the interaction between the agent and the environment. In the real-time market declaration strategy problem of the integrated energy system studied in this application, the execution effect of the declaration strategy is the actual clearing operating revenue. In the integrated energy system, the system operator's revenue can be expressed as the difference between the sales revenue of various types of energy on the user side and its own operating costs and market-side transaction fees, i.e.:

[0029] In the formula, The operational benefits corresponding to the winning bid results of the intelligent agent application strategy. For integrated energy system operators, the sales revenue of various types of energy, , These are the purchase costs of various types of energy and the operating costs of the integrated energy system network.

[0030] The revenue from the sale of various types of energy by the integrated energy system operator includes the revenue from the supply of electricity, gas, heat, and cooling to users. The purchase cost of each type of energy represents the cost of energy procured by the integrated energy system from the power grid, gas grid, and heating network. The power grid purchase cost is the cumulative market clearing price and the cleared electricity. The operating cost mainly considers the operating costs of combined cooling, heating, and power (CCHP), energy storage devices, and electric heating devices, which can be expressed as a quadratic function of energy consumption. The above three indicators can be expressed as follows:

[0031] In the formula, , These refer to the interval and number of periods for clearing out the spot market. , , , These are the user-side time periods of the integrated energy system. Electricity consumption, gas flow rate, heating power and cooling power; , , , These are the electricity price, gas price, heating price, and cooling price on the user side of the integrated energy system, and are generally fixed prices. , , These are the power purchased from the grid side of the integrated energy system, the gas flow rate purchased, and the heat purchased. , , These are the grid-side electricity purchase price, gas purchase price, and heat purchase price, respectively. , , These represent the operating costs of combined cooling, heating and power (CCHP), energy storage devices, and electric heating devices, respectively; represents the gas consumption flow rate of CCHP, represents the net exchange power of the energy storage device, and represents the power consumption of the electric heating device. , , , , , , , , The coefficients of the quadratic, linear, and constant terms of the operating cost function for combined cooling, heating, and power (CCHP), energy storage devices, and electric heating devices.

[0032] Considering the varying daily market transaction volumes and different base figures for operational returns, horizontal comparability is lacking. Therefore, this application proposes a decision ideality index as the reward function for the reporting strategy agent. This index is essentially a standardized result of the reporting strategy's returns. This application defines D as the ratio of the reporting decision return to the optimal solution, where the optimal solution is the optimal operational efficiency based on a given clearing price, which can be calculated using formulas [6-7]. In summary, the reward function can be expressed as:

[0033] In the formula, The reward function for decision-making ideality proposed in this application is... The optimal operating efficiency value is, where, The calculation method for the parameters is as follows: Based on historical data, clustering is performed using electricity load and price curves to form a set of typical application scenarios S, and each scenario is further defined based on the clustering results. probability of occurrence ;

[0034] For each scenario i, given the electricity price, calculate the corresponding revenue of each strategy in the strategy library, and select the strategy with the highest revenue. (The calculation formula is formula 4-7 from the patent). Optimal operational efficiency .

[0035] For step S3, in this embodiment of the application, obtaining the application scheme by filtering the optimal strategy from the application strategy library based on the optimal matching analysis includes: Based on the basic operational data of the integrated energy system, the application scheme is obtained by training the constructed intelligent agent model of the application strategy using a greedy strategy.

[0036] In this embodiment, the training process using the Q-reinforcement learning algorithm is essentially a process of updating and improving the agent's proposed strategy based on the action-state value function. The action-state value function is the expected reward corresponding to different actions taken by the agent, and can be expressed as:

[0037] In the formula, , For environment variables before and after the action is executed, , For environment variables , The actions that the next agent will take In environment variables The following actions are adopted. Transitioning environmental variables to The reward function below, , These are the action-state value functions before and after training and update, respectively. This is a discount factor to prevent premature convergence of the algorithm. To update the step size.

[0038] After the action-state value function is updated, the action policy is updated accordingly to the action that maximizes the expected action-state value function under that environment variable, which can be expressed as:

[0039] In the formula, For environment variables Next, agent action strategy, For environment variables The action option with the largest action-state value function among all available actions.

[0040] At the same time, to avoid premature development in action strategy selection, adopt A greedy strategy involves generating random numbers during the action selection phase. If random numbers Greater than the given limit Then, according to the agent's action strategy Output the action; otherwise, output the action space with equal probability. Choose one action from the options to output.

[0041] As described above, training is complete when an agent completes a given number of training updates, and its input data can be adjusted to the test dataset. If the reward function of the agent meets the convergence criteria during the testing phase, it indicates that the agent meets the requirements and can be deployed; otherwise, the agent can be improved by adjusting environmental variables, action space, etc., until a suitable agent for the application strategy is obtained.

[0042] Please refer to Table 1. The province's power sources are mainly thermal power, wind power, and photovoltaic power, with a total installed capacity of 903 billion kilowatts. Among them, thermal power accounts for 62 million kilowatts, while wind power and photovoltaic power each exceed 10 million kilowatts. The province started trial operation in 2019 and has conducted multiple consecutive trial runs, accumulating rich basic data on spot market transaction clearing.

[0043] Table 1

[0044] In this application embodiment, the key points of constructing a dynamic reporting decision-making agent for an integrated energy system include environmental variables, action space, and reward function. As shown in Table 2, the agent's action space is essentially a reporting strategy library. To meet the needs of dynamic reporting strategy decision-making for the agent, the reporting strategies in the reporting strategy library should comprehensively cover reporting calculations for different technical paths and different modeling methods. Therefore, the reporting strategy library design in this application comprehensively selects eight reporting strategies under paths such as current price taker, pricing participant, and two-layer architecture. It is particularly important to note that reporting strategies under the same technical path differ significantly in terms of modeling assumptions and modeling methods to ensure the effectiveness of the method proposed in this application.

[0045] Table 2

[0046] In this embodiment, a basic dataset for agent training, testing, and application will be constructed based on trial operation data from the spot market from January 1, 2020 to December 31, 2021. To ensure the effectiveness of the test and application datasets, one day will be randomly selected from each month of the aforementioned data to construct the test and application datasets. Thus, the training set comprises 706 days, while the test and application sets each comprise 24 days.

[0047] (1) Agent training and testing Specifically, the agent is trained and tested according to the reinforcement learning training method described above in this application, and the trend of its reward function is as follows: Figure 3 As shown. Figure 3 The bar chart shows the trend of the reward function during the training process. The range of the bar chart corresponds to the range of the decision ideality value in each round of training. It can be seen that the reinforcement learning agent has high training convergence efficiency and convergence accuracy. After about 10 rounds of iterative training, the agent's decision ideality is stably maintained at over 90%. Figure 4 The test results show that the average decision-making ideality of the 24 test subjects reached 98.5%, and the minimum reached 92.8%. These results indicate that the agent's decision-making accuracy and stability meet the application requirements, and it can be moved to the application stage.

[0048] (2) Application of intelligent agents In this embodiment of the application, the application set calculation result is as follows: Figure 5 As shown in the figure, it not only displays the decision-making ideality of the proposed application strategy but also the decision-making ideality of eight basic application strategies. It can be seen that the decision-making ideality of different basic application strategies varies significantly under different application scenarios, indicating that the applicability of different application strategies varies across scenarios. The dynamic application decision-making approach proposed in this paper accurately selects the application strategy with the highest decision-making ideality in the application set, ensuring the effectiveness of the application plan.

[0049] In this embodiment, by mining and analyzing historical data, the reinforcement learning agent can establish an adaptive matching relationship between application scenarios and declaration strategies, thereby rationally selecting the optimal declaration strategy, which can effectively improve the accuracy of declaration decisions in the spot market of integrated energy systems and enhance its market transaction efficiency.

[0050] Example 2 Figure 2 This is a diagram illustrating the apparatus for a market dynamic reporting strategy based on reinforcement learning, provided in one embodiment of this application. Please refer to... Figure 2 This application provides an apparatus for a market dynamic reporting strategy based on reinforcement learning, comprising: Module 101 is used to acquire basic operational data of the integrated energy system; The strategy construction module 102 is used to obtain the optimal matching analysis between strategies and scenarios based on the basic operating data of the integrated energy system; The strategy training module 103 is used to obtain the application scheme by selecting the optimal strategy from the application strategy library based on the optimal matching analysis.

[0051] Example 3 Figure 6 This is a schematic diagram of an electronic device provided in yet another embodiment of this application. Please refer to... Figure 6 This application provides an electronic device, including: At least one processor 31; and Memory 32 communicatively connected to the at least one processor; wherein, The memory 32 stores instructions that can be executed by the at least one processor 31, which, when executed by the at least one processor 31, enables the at least one processor 31 to perform the method described in any of the above embodiments.

[0052] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0053] It should be noted that in the description of this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means at least two.

[0054] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0055] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0056] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0057] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0058] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0059] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0060] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for market dynamic reporting strategies based on reinforcement learning, characterized in that, The method includes: Obtain basic operational data for integrated energy systems; Based on the basic operational data of the integrated energy system, a smart agent model for the application strategy is constructed to obtain the optimal matching analysis between the strategy and the scenario; Based on the aforementioned optimal matching analysis, the optimal strategy is selected from the application strategy library to obtain the application scheme; Constructing the intelligent agent model for the application strategy includes: By using the Q-reinforcement learning algorithm, we designed its three elements: environmental variables, action space, and reward function, and obtained the optimal matching analysis between policy scenarios. The environmental variables include: market transaction variables, including: benchmark power generation demand, new energy power generation forecast, conventional unit operating capacity, and fuel price index; The category of self-characteristics includes: self-electricity demand, self-heating demand, self-cooling demand, self-new energy power generation forecast, and self-energy storage capacity. The action space includes: A set of application strategies for integrated energy systems is obtained through agent-based decision-making. Based on the obtained set of integrated energy system declaration strategies, the optimal declaration strategy under the trading scenario described by the environmental variables is obtained. The formula for obtaining the optimal reporting strategy under the trading scenario described by the environmental variables is: In the formula, The number of selectable declaration strategies in the declaration strategy library within the action space is the integrated energy system declaration strategy set. , ... These are action status variables that sequentially indicate whether the 1st, 2nd, to Nth reporting strategies have been selected, with values ​​of 0 or 1. When the value is 1, it indicates that the reporting strategy is selected; Otherwise, it indicates that the strategy was not selected.

2. The method according to claim 1, characterized in that, The step of obtaining a declaration scheme by selecting the optimal strategy from the declaration strategy library based on the optimal matching analysis includes: Based on the basic operational data of the integrated energy system, the application scheme is obtained by training the constructed intelligent agent model of the application strategy using a greedy strategy.

3. An apparatus for a market dynamic reporting strategy based on reinforcement learning, used to execute the method of the market dynamic reporting strategy based on reinforcement learning as described in any one of claims 1-2, characterized in that, The device includes: The acquisition module is used to acquire basic operational data of the integrated energy system; The strategy construction module is used to obtain the optimal matching analysis between strategies and scenarios based on the basic operational data of the integrated energy system. The strategy training module is used to obtain the application scheme by selecting the optimal strategy from the application strategy library based on the optimal matching analysis.

4. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-2.

Images