Intelligent industrial park energy management system and method thereof

Abstract

The invention discloses an intelligent industrial park energy management system, which comprises a park energy management module, a park energy supply and demand interaction module and a park energy regulation and control module, and is characterized in that the park energy management module carries out supply and demand information sharing, data recording and transaction management on distributed energy in a park based on a block chain mode; the park energy supply and demand interaction module integrates distributed energy in a park and controllable loads of an external network through a virtual power plant architecture; the park energy regulation and control module regulates and controls coordination of park distributed energy and external network power resources based on a deep reinforcement learning network, and the distributed energy, the energy storage system and the load resources in the park can be aggregated to participate in power grid dispatching in an overall mode. And the block chain provides a transparent and credible transaction platform through decentration.

Description

Technical Field

[0001] The present invention relates to the field of electrical engineering, and in particular to an intelligent industrial park energy management system and method thereof. Background Art

[0002] At present, the energy demand and management mode of industrial parks are undergoing profound changes. Industrial parks usually have large energy load demands and are also ideal places for distributed energy applications, including renewable energy such as photovoltaics and wind power, as well as energy storage systems. With the widespread application of distributed energy, how to achieve efficient energy scheduling, balance supply and demand, and maximize economic benefits has become a key issue in energy management in industrial parks.

[0003] At present, the traditional power system dispatching method is difficult to adapt to the complex energy structure and dynamic load demand of industrial parks. In order to meet this challenge, researchers have begun to explore intelligent energy management solutions based on blockchain and deep reinforcement learning in recent years. The decentralization, immutability and smart contract technology of blockchain provide a safe and transparent trading platform for the trading and management of distributed energy; while deep reinforcement learning can realize the adaptive regulation of complex systems, which has significant advantages in supply and demand balance and load fluctuations.

[0004] Domestic and foreign research on industrial park power systems and distributed energy management focuses on the following aspects: the application of blockchain in energy systems, virtual power plant (VPP) control strategies, and adaptive supply and demand management based on deep reinforcement learning.

[0005] In terms of the application of blockchain in energy systems, Wang et al. proposed a blockchain-based energy management system to support distributed energy transactions and supply and demand interactions within the park. Kim and Yang studied the application of blockchain in microgrid networks and built a distributed energy market to improve the marketization of energy and resource utilization. The existing technology proposed a distributed power trading platform based on blockchain, which uses smart contracts to automatically execute transactions, significantly improving the efficiency and security of energy management.

[0006] In terms of virtual power plant technology, foreign scholars have made a lot of progress in the research of virtual power plants (VPP), especially in the coordinated dispatch of multiple energy sources and load tracking. At the same time, the existing technology proposes a VPP architecture suitable for industrial parks, which can integrate distributed energy, energy storage systems and adjustable load resources to form an overall virtual power plant to participate in power market transactions and achieve efficient resource dispatch and optimization.

[0007] In the application of deep reinforcement learning in intelligent regulation, in the field of deep reinforcement learning (DRL), Chen et al. used DRL to build a flexible demand response model, which enables the smart grid system to show strong adaptability and flexibility when facing load demand fluctuations. At the same time, the existing technology uses reinforcement learning models to achieve adaptive control of energy storage systems, allowing the system to discharge during peak periods and charge during valley periods, thereby reducing overall operating costs. Summary of the invention

[0008] In view of the technical problems existing in the prior art, the present invention provides an intelligent industrial park energy management system and method thereof. The invention designs a distributed energy management system (DEMS) based on blockchain and combines a multi-agent reinforcement learning mechanism to improve the adaptability and response speed of the park power system. The virtual power plant designed by the present invention can aggregate the distributed energy, energy storage system and load resources in the park and participate in the grid dispatching in an integrated manner, while the blockchain provides a transparent and reliable trading platform through decentralization.

[0009] In order to solve the technical problem, the present invention is implemented by adopting the following technical solution:

[0010] An intelligent industrial park energy management system, the system includes a park energy management module, a park energy supply and demand interaction module and a park energy control module, wherein:

[0011] The park energy management module is based on blockchain to share supply and demand information, record data and manage transactions of distributed energy within the park;

[0012] The park energy supply and demand interaction module integrates the distributed energy in the park and the controllable load of the external network through the virtual power plant architecture;

[0013] The park energy control module coordinates the park's distributed energy and external network power resources based on a deep reinforcement learning network.

[0014] Furthermore, the park energy management module manages the distributed energy transaction process in the park through price contracts and internal energy transactions, including:

[0015] According to the demand of user i in the park, and the distributed energy generation capacity is The park transaction power is constructed according to the following formula:

[0016]

[0017] Among them: When it indicates the purchase demand, When , it indicates surplus;

[0018] Based on the market electricity price t The price contract within the park is constructed according to the following formula:

[0019] p t =λ t (1+δ)

[0020] Among them: δ is the incentive factor, which is used to encourage transactions within the park and reduce the load on the power grid.

[0021] Furthermore, the virtual power plant architecture is composed of a park level and a grid level; the park energy supply and demand interaction module integrates distributed energy and controllable loads of the external network, including:

[0022] 201. Under the dynamic balance of the energy state space and load demand of the park, the power supply and demand balance model of the park is constructed by the following formula:

[0023]

[0024] And constrain the power upper limit of each variable:

[0025]

[0026] Where: D t The total load demand of the park at time t; is the power generation of distributed energy; the total output of photovoltaic and wind power in the park, The amount of electricity provided by the energy storage module, the amount of charge and discharge of the energy storage system at time t, a positive value indicates charging, and a negative value indicates discharging; the power purchased by the grid is

[0027] 202. The following formula calculates the grid purchase cost, energy storage system cost and distributed energy operation cost to construct the economic cost model of the park operation, including:

[0028] Cost of purchasing electricity from the power grid:

[0029]

[0030] Among them, λ t Indicates the electricity price.

[0031] Energy storage system cost:

[0032]

[0033] Among them, α is the charging and discharging cost of the energy storage system.

[0034] Distributed energy costs:

[0035]

[0036] The total economic cost is:

[0037]

[0038] Furthermore, the park energy control module controls the coordination process of the park distributed energy and external network power resources based on the deep reinforcement learning network, including:

[0039] 301. Construct a park energy control model based on reinforcement learning state parameters and action time parameters, including:

[0040] The state space S t Contains current load demand D t , Distributed Energy Output Energy storage system status SOC t and electricity price λ t ; The action time A t Includes adjustment of energy storage charging and discharging, distributed energy output and load priority;

[0041] 302. Optimize the park energy control model through the reward function to build a park energy adaptive control model:

[0042] Reward function R t Balance operating costs and supply-demand differences to achieve supply-demand balance and cost minimization:

[0043]

[0044] Among them, γ is the penalty factor, and the reward function R t Balancing operating costs and supply-demand differences;

[0045] 303. The park energy adaptive control model is updated through the following formula:

[0046]

[0047] Among them, α represents the learning rate, which is used to control the speed of updating; γ is the discount factor, which balances the weights of current and future rewards; the Q value represents the expected cumulative reward that the agent can obtain from the current state to the future terminal state after taking action A in state S; the cumulative reward includes not only the immediate reward at the current moment, but also the accumulated value of the discounted reward at future moments.

[0048] The present invention can also be implemented by the following technical solution, wherein the method comprises the following steps:

[0049] S1. Sharing supply and demand information, data recording and transaction management of distributed energy in the park based on blockchain; according to the demand of user i in the park and the distributed energy generation capacity is The park transaction power is constructed according to the following formula:

[0050]

[0051] Among them: When it indicates the purchase demand, When , it indicates surplus;

[0052] Based on the market electricity price t The price contract within the park is constructed according to the following formula:

[0053] p t =λ t (1+δ)

[0054] Among them: δ is the incentive factor, which is used to encourage internal transactions in the park and reduce the load on the power grid;

[0055] S2: The process of integrating distributed energy and controllable loads of the external network includes:

[0056] 201. Under the dynamic balance of the energy state space and load demand of the park, the power supply and demand balance model of the park is constructed by the following formula:

[0057]

[0058] And constrain the power upper limit of each variable:

[0059]

[0060] Where: D t The total load demand of the park at time t; is the power generation of distributed energy; the total output of photovoltaic and wind power in the park, The amount of electricity provided by the energy storage module, the amount of charge and discharge of the energy storage system at time t, a positive value indicates charging, and a negative value indicates discharging; the power purchased by the grid is

[0061] 202. The following formula calculates the grid purchase cost, energy storage system cost and distributed energy operation cost to construct the economic cost model of the park operation, including:

[0062] Cost of purchasing electricity from the power grid:

[0063]

[0064] Among them, λ t Indicates the price of electricity;

[0065] Energy storage system cost:

[0066]

[0067] Among them, α is the charging and discharging cost of the energy storage system;

[0068] Distributed energy costs:

[0069]

[0070] The total economic cost is:

[0071]

[0072] S3. The coordination process of the distributed energy resources in the park and the external power resources based on deep reinforcement learning network control includes:

[0073] 301. Construct a park energy control model based on reinforcement learning state parameters and action time parameters, including:

[0074] The state space S t Contains current load demand D t , Distributed Energy Output E t gen , Energy storage system status SOC t and electricity price λ t ; The action time A t Includes adjustment of energy storage charging and discharging, distributed energy output and load priority;

[0075] 302. Optimize the park energy control model through the reward function to build a park energy adaptive control model:

[0076] Reward function R t Balance operating costs and supply-demand differences to achieve supply-demand balance and cost minimization:

[0077]

[0078] Among them, γ is the penalty factor, and the reward function R t Balancing operating costs and supply-demand differences;

[0079] 303. The park energy adaptive control model is updated through the following formula:

[0080]

[0081] Among them, α represents the learning rate, which is used to control the speed of updating; γ is the discount factor, which balances the weights of current and future rewards; the Q value represents the expected cumulative reward that the agent can obtain from the current state to the future terminal state after taking action A in state S; the cumulative reward includes not only the immediate reward at the current moment, but also the accumulated value of the discounted reward at future moments.

[0082] Beneficial Effects

[0083] The beneficial effects of the technical solution provided by the present invention are:

[0084] This paper uses smart contracts, virtual power plants and multi-agent reinforcement learning mechanisms to improve the coordinated regulation of power resources inside and outside the park. Experimental and simulation results show that the proposed method is significantly superior to traditional methods in terms of economic benefits, clean energy utilization and regulation flexibility.

[0085] 1) Improved economic benefits: Through smart contracts and optimized scheduling of energy storage systems, the total cost of the park within a 24-hour cycle is significantly reduced. During peak periods of electricity price fluctuations, the optimization solution can reasonably arrange the charging and discharging strategy of the energy storage system, thereby reducing the demand for electricity purchases from the power grid during peak electricity prices. The comparison data is: the total cost of the traditional method is an average of 321 yuan in each period, and the highest cost during peak periods is 520 yuan. Optimization method: The total cost is an average of 273 yuan in each period, and the highest cost during peak periods is reduced to 450 yuan, with an overall cost saving of about 18%.

[0086] 2) Improvement of clean energy utilization: The optimization scheme significantly improves the utilization rate of clean energy in the park by rationally scheduling the use of distributed energy, especially during periods of high load. Point-to-point energy sharing is achieved through smart contracts, further reducing dependence on the power grid. The comparison results of clean energy utilization are as follows: Traditional method: The average clean energy utilization rate is about 47%. Optimization method: The average clean energy utilization rate is increased to 63%, and it is increased by about 20% during peak load periods, reaching 80% utilization.

[0087] 3) Improved control flexibility and response speed: The adaptive control module driven by reinforcement learning enables the system to quickly adjust the SOC state of the energy storage system when facing load and electricity price fluctuations. Through adaptive control of the charging and discharging of the energy storage system, the optimization scheme shows significant improvements in response time and adaptability. The specific data are as follows: Traditional method: The system response time is an average of 30 minutes when electricity prices and loads fluctuate. Optimization method: Through reinforcement learning adaptive control, the system response time is shortened to 21 minutes, a reduction of about 30%. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 This is a flow chart of an intelligent industrial park energy management system of the present invention;

[0089] Figure 2 A total cost change trend diagram of an intelligent industrial park energy management system according to the present invention;

[0090] Figure 3 This is a state change diagram of an energy storage module of an intelligent industrial park energy management system of the present invention;

[0091] Figure 4 This is a chart showing the improvement in clean energy utilization. DETAILED DESCRIPTION

[0092] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.

[0093] like Figure 1 As shown, the present invention provides an intelligent industrial park energy management system, which is a supply and demand interaction optimization method based on blockchain and deep reinforcement learning. The system includes a park energy management module, a park energy supply and demand interaction module and a park energy regulation module, wherein:

[0094] The park energy management module (DEMS) is based on blockchain to share supply and demand information, record data and manage transactions for distributed energy in the park; the park energy management module of the present invention builds energy management and transactions within the park based on blockchain. Distributed energy (such as photovoltaic, wind power, energy storage system, etc.) in the park is registered through blockchain, and smart contracts are used to achieve supply and demand information sharing, data recording and transaction management. The design of this system can ensure the transparency and immutability of data, and reduce the management cost and security risks of traditional centralized systems.

[0095] The park energy management module manages the distributed energy transaction process in the park through price contracts and internal energy transactions, including:

[0096] According to the demand of user i in the park, and the distributed energy generation capacity is The park transaction power is constructed according to the following formula:

[0097]

[0098] Among them: When it indicates the purchase demand, When , it indicates surplus;

[0099] Based on the market electricity price t The price contract within the park is constructed according to the following formula:

[0100] p t =λ t(1+δ)

[0101] Among them: δ is the incentive factor, which is used to encourage transactions within the park and reduce the load on the power grid.

[0102] The park energy supply and demand interaction module integrates the distributed energy in the park and the controllable load of the external network through the virtual power plant architecture (VPP); the virtual power plant architecture is composed of the park level and the grid level; the park energy supply and demand interaction module integrates the distributed energy and the controllable load of the external network, including:

[0103] 201. Under the dynamic balance of the energy state space and load demand of the park, the power supply and demand balance model of the park is constructed by the following formula:

[0104]

[0105] And constrain the power upper limit of each variable:

[0106]

[0107] Where: D t The total load demand of the park at time t; is the power generation of distributed energy; the total output of photovoltaic and wind power in the park, The amount of electricity provided by the energy storage module, the amount of charge and discharge of the energy storage system at time t, a positive value indicates charging, and a negative value indicates discharging; the power purchased by the grid is

[0108] 202. The following formula calculates the grid purchase cost, energy storage system cost and distributed energy operation cost to construct the economic cost model of the park operation, including:

[0109] Cost of purchasing electricity from the power grid:

[0110]

[0111] Among them, λ t Indicates the electricity price.

[0112] Energy storage system cost:

[0113]

[0114] Among them, α is the charging and discharging cost of the energy storage system.

[0115] Distributed energy costs:

[0116]

[0117] The total economic cost is:

[0118]

[0119] The park energy control module is based on a deep reinforcement learning network to control the coordination of the park's distributed energy and external power resources; the park energy control module is based on an adaptive control method based on multi-agent reinforcement learning, using algorithms such as deep Q network (DQN) and proximal policy optimization (PPO), and each agent can adaptively learn and optimize its own control strategy to cope with dynamic demand changes. This design allows the system to maintain flexibility and robustness in the face of external environment and demand fluctuations. Among them:

[0120] The park energy control module controls the coordination process of the park distributed energy and external network power resources based on the deep reinforcement learning network, including:

[0121] 301. Construct a park energy control model based on reinforcement learning state parameters and action time parameters, including:

[0122] The state space S t Contains current load demand D t , Distributed Energy Output Energy storage system status SOC t and electricity price λ t ; The action time A t Includes adjustment of energy storage charging and discharging, distributed energy output and load priority;

[0123] 302. Optimize the park energy control model through the reward function to build a park energy adaptive control model:

[0124] Reward function R t Balance operating costs and supply-demand differences to achieve supply-demand balance and cost minimization:

[0125]

[0126] Among them, γ is the penalty factor, and the reward function R t Balancing operating costs and supply-demand differences;

[0127] 303. The park energy adaptive control model is updated through the following formula:

[0128]

[0129] Among them, α represents the learning rate, which is used to control the speed of updating; γ is the discount factor, which balances the weights of current and future rewards; the Q value represents the expected cumulative reward that the agent can obtain from the current state to the future terminal state after taking action A in state S; the cumulative reward includes not only the immediate reward at the current moment, but also the accumulated value of the discounted reward at future moments.

[0130] In summary, the embodiments of the present invention, through the above method, are significantly superior to traditional methods in terms of flexibility, energy utilization and economic benefits, and demonstrate broad application potential in future intelligent industrial park energy management.

[0131] The feasibility of the scheme is verified by combining specific experimental data. See the following description for details:

[0132] The examples describe in detail the setting of the simulation experiment, the selection of software, the parameter configuration and the result analysis to verify the application effect of the proposed supply-demand interactive optimization method based on blockchain and deep reinforcement learning in the power system of the industrial park. The experiment simulates the power demand, electricity price fluctuations and distributed energy generation for 24 hours a day by using MATLAB / Simulink and Python to evaluate the performance of the optimization scheme in terms of economic benefits, clean energy utilization and response time.

[0133] Step S1: Experimental setup

[0134] In the simulation experiment, a 24-hour simulation scenario was constructed. Based on the real industrial park load demand, electricity price and distributed energy data, different experimental parameters were set to verify the supply and demand interaction optimization method proposed in this paper. The experiment mainly examined the following aspects:

[0135] Flexibility: Measures the system's response speed and adjustment capabilities to fluctuations in load demand and electricity prices.

[0136] Economic benefits: Calculate the total operating cost of the park through optimization strategies and compare the savings of traditional control methods.

[0137] Clean energy utilization rate: Maximize the use of distributed energy and reduce power purchases from the grid without affecting the balance between supply and demand.

[0138] During the experiment, we set different charging and discharging strategies for the energy storage system and introduced random fluctuations in wind power generation to observe the performance of the system under uncertain conditions.

[0139] Step S2: Parameter setting

[0140] Table 1 Parameter settings

[0141]

[0142] Step S3: Scene description

[0143] Assume that in a 24-hour period, the industrial park experiences the following typical operating scenarios:

[0144] ① Daytime peak period (9 a.m. to 5 p.m.): Load demand is high, photovoltaic power generation is large, and electricity prices are at a peak. The energy storage system can discharge to reduce power purchases from the grid.

[0145] ② Nighttime off-peak period (10 p.m. to 6 a.m. the next day): Load demand is low, electricity prices are low, and the energy storage system is charged to release electricity during peak hours.

[0146] ③ Random fluctuations in wind power generation: Wind power output is uncertain, and its fluctuation characteristics are modeled through actual wind speed data.

[0147] Step S4: Experimental steps

[0148] Step 401: Environment initialization:

[0149] Initial load demand, distributed energy generation, electricity price, and energy storage SOC are set through MATLAB.

[0150] Step 402: Reinforcement learning training:

[0151] Use Python (TensorFlow, PyTorch) to train the strategy based on the DQN algorithm within 24 hours and determine the optimal control solution.

[0152] Step 403: Smart Contract Execution:

[0153] Use blockchain to simulate the automatic execution of smart contracts, record the transaction process of excess electricity and update the power purchase cost of the park.

[0154] Step 404: Data recording and analysis:

[0155] Load demand, energy storage SOC, clean energy utilization, power purchase from the grid and total operating cost are recorded in each time period.

[0156] Table 2 Simulation results

[0157]

[0158] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0159] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. An intelligent industrial park energy management system, characterized in that: The system includes a park energy management module, a park energy supply and demand interaction module and a park energy control module, wherein: The park energy management module is based on blockchain to share supply and demand information, record data and manage transactions of distributed energy within the park; The park energy supply and demand interaction module integrates the distributed energy in the park and the controllable load of the external network through the virtual power plant architecture; The park energy control module coordinates the park's distributed energy and external network power resources based on a deep reinforcement learning network.

2. According to claim 1, an intelligent industrial park energy management system is characterized in that: The park energy management module manages the distributed energy transaction process in the park through price contracts and internal energy transactions, including: According to the demand of user i in the park, and the distributed energy generation capacity is The park transaction power is constructed according to the following formula: Among them: When it indicates the purchase demand, When , it indicates surplus; Based on the market electricity price t The price contract within the park is constructed according to the following formula: p t =λ t (1+d) Among them: δ is the incentive factor, which is used to encourage transactions within the park and reduce the load on the power grid.

3. The intelligent industrial park energy management system according to claim 1 is characterized in that: The virtual power plant architecture consists of a park level and a grid level. The park energy supply and demand interaction module integrates distributed energy and controllable loads of the external network, including:

201. Under the dynamic balance of the energy state space and load demand of the park, the power supply and demand balance model of the park is constructed by the following formula: And constrain the power upper limit of each variable: Where: D t The total load demand of the park at time t; is the power generation of distributed energy; the total output of photovoltaic and wind power in the park, The amount of electricity provided by the energy storage module, the amount of charge and discharge of the energy storage system at time t, a positive value indicates charging, and a negative value indicates discharging; the power purchased by the grid is 202. The following formula calculates the grid purchase cost, energy storage system cost and distributed energy operation cost to construct the economic cost model of the park operation, including: Cost of purchasing electricity from the power grid: Among them, λ t Indicates the electricity price. Energy storage system cost: Among them, α is the charging and discharging cost of the energy storage system. Distributed energy costs: The total economic cost is: 。 4. The intelligent industrial park energy management system according to claim 1 is characterized in that: The park energy control module controls the coordination process of the park distributed energy and external network power resources based on the deep reinforcement learning network, including:

301. Construct a park energy control model based on reinforcement learning state parameters and action time parameters, including: The state space S t Contains current load demand D t , Distributed Energy Output Energy storage system status SOC t and electricity price λ t ; The action time A t Includes adjustment of energy storage charging and discharging, distributed energy output and load priority; 302. Optimize the park energy control model through the reward function to build a park energy adaptive control model: Reward function R t Balance operating costs and supply-demand differences to achieve supply-demand balance and cost minimization: Among them, γ is the penalty factor, and the reward function R t Balancing operating costs and supply-demand differences; 303. The park energy adaptive control model is updated through the following formula: Among them, α represents the learning rate, which is used to control the speed of updating; γ is the discount factor, which balances the weights of current and future rewards; the Q value represents the expected cumulative reward that the agent can obtain from the current state to the future terminal state after taking action A in state S; the cumulative reward includes not only the immediate reward at the current moment, but also the accumulated value of the discounted reward at future moments.

5. The method for intelligently managing energy in an industrial park according to claim 1, characterized in that: The method comprises the following steps: S1. Share supply and demand information, record data and manage transactions of distributed energy in the park based on blockchain. According to the demand of user i in the park, D t i and the distributed energy generation capacity is The park transaction power is constructed according to the following formula: Among them: When it indicates the purchase demand, When , it indicates surplus; Based on the market electricity price t The price contract within the park is constructed according to the following formula: p t =λ t (1+d) Among them: δ is the incentive factor, which is used to encourage internal transactions in the park and reduce the load on the power grid; S2: The process of integrating distributed energy and controllable loads of the external network includes:

201. Under the dynamic balance of the energy state space and load demand of the park, the power supply and demand balance model of the park is constructed by the following formula: And constrain the power upper limit of each variable: Where: D t The total load demand of the park at time t; is the power generation of distributed energy; the total output of photovoltaic and wind power in the park, The amount of electricity provided by the energy storage module, the amount of charge and discharge of the energy storage system at time t, a positive value indicates charging, and a negative value indicates discharging; the power purchased by the grid is 202. The following formula calculates the grid purchase cost, energy storage system cost and distributed energy operation cost to construct the economic cost model of the park operation, including: Cost of purchasing electricity from the power grid: Among them, λ t Indicates the price of electricity; Energy storage system cost: Among them, α is the charging and discharging cost of the energy storage system; Distributed energy costs: The total economic cost is: S3. The coordination process of the distributed energy resources in the park and the external power resources based on deep reinforcement learning network control includes:

301. Construct a park energy control model based on reinforcement learning state parameters and action time parameters, including: The state space S t Contains current load demand D t , Distributed Energy Output Energy storage system status SOC t and electricity price λ t ; The action time A t Includes adjustment of energy storage charging and discharging, distributed energy output and load priority; 302. Optimize the energy control model of the park through the reward function to build an adaptive energy control model of the park: reward function R t Balance operating costs and supply-demand differences to achieve supply-demand balance and cost minimization: Among them, γ is the penalty factor, and the reward function R t Balancing operating costs and supply-demand differences; 303. The park energy adaptive control model is updated through the following formula: Among them, α represents the learning rate, which is used to control the speed of updating; γ is the discount factor, which balances the weights of current and future rewards; the Q value represents the expected cumulative reward that the agent can obtain from the current state to the future terminal state after taking action A in state S; the cumulative reward includes not only the immediate reward at the current moment, but also the accumulated value of the discounted reward at future moments.

Images