Interconnected micro-grid energy scheduling method fusing expert knowledge and reinforcement learning technology

By integrating expert knowledge and reinforcement learning technology, combining knowledge-driven and data-driven layered design, optimizing the energy scheduling of the interconnected microgrid, the complexity and uncertainty of energy scheduling in large-scale scenarios is solved, and operating costs and stability are achieved The comprehensive optimization of the scheduling strategy has significantly improved the real-time and flexibility of the scheduling strategy.

CN118941052APending Publication Date: 2024-11-12HUNAN FIRST NORMAL UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411157222.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In large-scale scenarios, the energy scheduling of the Internet microgrid faces the problems of high training complexity, large exploration space and low training efficiency, and it is difficult to meet the practical application needs. Especially due to the intermittent and volatility of renewable energy, the scheduling strategy must cope with a high degree of Uncertainty.

Method used

By integrating expert knowledge and reinforcement learning technology, we will optimize the equipment start-stop and power distribution by establishing an interconnected microgrid reinforcement learning environment, introducing large strategic optimization modules, and performing layered optimization designs. Use constraints and optimization target modules for safe allocation to achieve iterative optimization and performance evaluation.

Benefits of technology

It significantly reduces operating costs, improves decision-making speed and system stability, reduces the average operating costs by 13.4%, improves decision-making stability by 18.9%, and shortens response time by 17.6%, achieving more efficient, economical and safe operation of power systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941052A_ABST
    Figure CN118941052A_ABST
Patent Text Reader

Abstract

The invention discloses an interconnected micro-grid energy scheduling method fusing expert knowledge and reinforcement learning technology, which belongs to the field of smart grids, and comprises the following steps: S1, establishing an interconnected micro-grid reinforcement learning environment, and introducing an optimization strategy model; s2, layered optimization design is carried out; s3, iterative optimization of the reinforcement learning strategy; and S4, performing model application and performance evaluation. According to the interconnected micro-grid energy scheduling method fusing expert knowledge and reinforcement learning technology, the training time of reinforcement learning is shortened, the robustness and stability of the strategy are enhanced, the operation cost of the interconnected micro-grid is further reduced by optimizing the energy scheduling strategy, and the energy scheduling efficiency is improved. And a key technical support is provided for realizing efficient, economical and safe operation of the large-scale interconnected micro-grid.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of smart grids, and in particular to an interconnected microgrid energy scheduling method that integrates expert knowledge and reinforcement learning technology. Background Art

[0002] As a new type of power network architecture, interconnected microgrids achieve effective energy sharing by connecting multiple adjacent microgrids, significantly improving the stability and economy of energy supply. Energy scheduling, as the core link of interconnected microgrids, aims to achieve a dynamic balance between power supply and demand by finely managing the power distribution of distributed power sources in each microgrid and the power flow between power grids, while minimizing the overall operating cost. This process plays an irreplaceable role in promoting the large-scale integration of renewable energy, optimizing energy utilization efficiency, and ensuring the safe and stable operation of the system.

[0003] However, energy scheduling in interconnected microgrids faces many challenges. The primary problem is the differences in cost characteristics, diversity in operating states, and complex operating constraints among microgrids. In addition, due to the intermittent and volatile nature of renewable energy generation, energy scheduling strategies must be able to cope with a high degree of uncertainty. Although energy scheduling strategies based on reinforcement learning have achieved certain results in recent years, this method faces difficulties such as high training complexity, large exploration space, and low training efficiency in large-scale interconnected microgrid scenarios, making it difficult to meet actual application needs. Summary of the invention

[0004] The present invention aims to provide an interconnected microgrid energy scheduling method that integrates expert knowledge and reinforcement learning technology to solve the problems raised in the above background technology.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for energy dispatching of interconnected microgrids integrating expert knowledge and reinforcement learning technology comprises the following steps:

[0007] S1. Establish an interconnected microgrid reinforcement learning environment and introduce a large strategy optimization module;

[0008] S2. Perform hierarchical optimization design on the strategy optimization module according to the energy scheduling problem of interconnected microgrids;

[0009] S3, iterative optimization of reinforcement learning strategies;

[0010] S4, model application and performance evaluation;

[0011] The established interconnected microgrid reinforcement learning environment includes a microgrid community and a strategy optimization module, wherein the microgrid community can input real-time status into the strategy optimization module and receive real-time optimization scheduling strategy from the strategy optimization module;

[0012] The strategy optimization module performs hierarchical optimization design for the energy scheduling problem of interconnected microgrids, including:

[0013] The upper-level modules include knowledge-driven modules and reinforcement learning algorithm modules, which optimize the start and stop of equipment.

[0014] The lower module consists of a constraint condition module and an optimization target module, which optimizes the power safety distribution;

[0015] The start-stop action obtained by the upper module is input to the lower module, and the lower module designs the optimal power allocation algorithm to obtain the safe power action; at the same time, in the lower module, the operating cost reward and safety penalty related to reinforcement learning are calculated and fed back to the upper reinforcement learning algorithm; the start-stop action of the upper module and the reward of the lower module interact with each other to adjust the start-stop action of the upper module and the reward of the lower module, so as to continuously optimize the reinforcement learning strategy of equipment start-stop;

[0016] The step S4 establishes an energy scheduling model obtained by training and optimizing according to S2 and S3, applies the energy scheduling model to the actual interconnected microgrid, monitors the operating status of the system, collects actual operating data, and evaluates the effectiveness and economy of the scheduling strategy.

[0017] Preferably, the microgrid community includes a plurality of microgrids;

[0018] Preferably, the knowledge-driven module includes an expert knowledge base and an optimized action space, and the optimized action space can call the expert knowledge base data.

[0019] Preferably, the constraint condition module has two preset conditions of equipment output limitation and energy storage equipment capacity constraint, and the optimization target module has four preset conditions of operation cost, transmission cost, startup cost and switching cost.

[0020] Preferably, the energy scheduling model is updated and stored in real time.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] The solution of the present invention significantly improves the intelligent level of energy dispatching of interconnected microgrids, realizes the comprehensive optimization of operating costs, decision-making speed and system stability, and provides strong technical support for building an efficient, economical and safe power system.

[0023] Significantly reduced operating costs: Compared with traditional reinforcement learning methods without expert knowledge, the knowledge-driven hierarchical security reinforcement learning interconnected microgrid energy scheduling optimization strategy proposed in this invention significantly reduces the operating costs of interconnected microgrids by accurately controlling the start and stop and power distribution of power generation equipment, reducing the operating costs by an average of more than 13.4% while maintaining higher system stability.

[0024] Speed ​​up scheduling decisions: By optimizing the action space, the training efficiency of the reinforcement learning algorithm is greatly improved. Compared with the traditional reinforcement learning method without expert knowledge, the decision stability is improved by about 18.9% and the response time is shortened by about 17.6% through the layered design and the addition of the safe action layer, which effectively improves the real-time and flexibility of the scheduling strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Flowchart of an interconnected microgrid energy scheduling method that integrates expert knowledge and reinforcement learning technology. DETAILED DESCRIPTION

[0026] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments:

[0027] The specific implementation process is as follows:

[0028] like Figure 1 As shown in FIG, an interconnected microgrid energy scheduling method integrating expert knowledge and reinforcement learning technology combines knowledge-driven expert experience and data-driven reinforcement learning to reduce the operating cost of the interconnected microgrid, including the following steps:

[0029] S1. Establish an interconnected microgrid reinforcement learning environment and introduce a large strategy optimization module;

[0030] S2. Perform hierarchical optimization design on the strategy optimization module according to the energy scheduling problem of interconnected microgrids;

[0031] S3, iterative optimization of reinforcement learning strategies;

[0032] S4, model application and performance evaluation;

[0033] The interconnected microgrid reinforcement learning environment established in step S1 includes a microgrid community and a strategy optimization module. The microgrid community can input real-time status into the strategy optimization module and receive real-time optimization scheduling strategies from the strategy optimization module. The microgrid community includes several microgrids.

[0034] The strategy optimization module is designed for the energy scheduling problem of interconnected microgrids in a hierarchical manner, including:

[0035] The upper-level module includes a knowledge-driven module and a reinforcement learning algorithm module, which optimize the start and stop of the equipment. The knowledge-driven module includes an expert knowledge base and an optimized action space. The optimized action space can call on the expert knowledge base data.

[0036] The lower module consists of a constraint condition module and an optimization target module, which optimizes the safe power distribution. The constraint condition module has two preset conditions: equipment output limit and energy storage equipment capacity constraint. The optimization target module has four preset conditions: operating cost, transmission cost, startup cost and switching cost.

[0037] The start-stop actions obtained by the upper module are input to the lower module. The lower module designs the optimal power allocation algorithm to obtain safe power actions. At the same time, in the lower module, the operating cost rewards and safety penalties related to reinforcement learning are calculated and fed back to the upper reinforcement learning algorithm. The start-stop actions of the upper module and the rewards of the lower module interact and adjust each other, thereby continuously optimizing the reinforcement learning strategy for equipment start-stop;

[0038] Step S4 establishes an energy scheduling model obtained by training and optimization according to S2 and S3, applies the energy scheduling model to the actual interconnected microgrid, monitors the operating status of the system, collects actual operating data, evaluates the effectiveness and economy of the scheduling strategy, and makes necessary adjustments and optimizations to the model according to the actual operating conditions to ensure its stability and reliability under different operating conditions.

[0039] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions and / or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical solution of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. An interconnected microgrid energy scheduling method integrating expert knowledge and reinforcement learning technology, characterized in that: The steps include: S1. Establish an interconnected microgrid reinforcement learning environment and introduce a large strategy optimization module; S2. Perform hierarchical optimization design on the strategy optimization module according to the energy scheduling problem of interconnected microgrids; S3, iterative optimization of reinforcement learning strategies; S4, model application and performance evaluation; The established interconnected microgrid reinforcement learning environment includes a microgrid community and a strategy optimization module, wherein the microgrid community can input real-time status into the strategy optimization module and receive real-time optimization scheduling strategy from the strategy optimization module; The strategy optimization module performs hierarchical optimization design for the energy scheduling problem of interconnected microgrids, including: The upper-level modules include knowledge-driven modules and reinforcement learning algorithm modules, which optimize the start and stop of equipment. The lower module consists of a constraint condition module and an optimization target module, which optimizes the power safety distribution; The start-stop action obtained by the upper module is input to the lower module, and the lower module designs the optimal power allocation algorithm to obtain the safe power action; at the same time, in the lower module, the operating cost reward and safety penalty related to reinforcement learning are calculated and fed back to the upper reinforcement learning algorithm; the start-stop action of the upper module and the reward of the lower module interact with each other to adjust the start-stop action of the upper module and the reward of the lower module, so as to continuously optimize the reinforcement learning strategy of equipment start-stop; The step S4 establishes an energy scheduling model obtained by training and optimizing according to S2 and S3, applies the energy scheduling model to the actual interconnected microgrid, monitors the operating status of the system, collects actual operating data, and evaluates the effectiveness and economy of the scheduling strategy.

2. The interconnected microgrid energy dispatching method integrating expert knowledge and reinforcement learning technology according to claim 1 is characterized by: The microgrid cluster includes a plurality of microgrids.

3. The interconnected microgrid energy dispatching method integrating expert knowledge and reinforcement learning technology according to claim 1 is characterized in that: The knowledge-driven module includes an expert knowledge base and an optimized action space, and the optimized action space can call the expert knowledge base data.

4. The interconnected microgrid energy dispatching method integrating expert knowledge and reinforcement learning technology according to claim 1 is characterized in that: The constraint condition module is preset with two conditions: equipment output limitation and energy storage equipment capacity constraint. The optimization target module is preset with four conditions: operation cost, transmission cost, startup cost and switching cost.

5. The interconnected microgrid energy dispatching method integrating expert knowledge and reinforcement learning technology according to claim 1 is characterized by: The energy scheduling model is updated and stored in real time.