Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Power grid energy management method and system based on deep expectation Q-learning

An energy management and learning network technology, applied in the field of power grid energy management system, can solve problems such as difficulty in obtaining the global optimal solution of nonlinear optimization models, and long time-consuming heuristic optimization algorithms.

Active Publication Date: 2021-04-06
STATE GRID SICHUAN ELECTRIC POWER CORP ELECTRIC POWER RES INST
View PDF6 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

It is difficult for classical optimization algorithms to obtain the global optimal solution of this type of nonlinear optimization model, while heuristic optimization algorithms generally take a long time

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Power grid energy management method and system based on deep expectation Q-learning
  • Power grid energy management method and system based on deep expectation Q-learning
  • Power grid energy management method and system based on deep expectation Q-learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0151] Such as figure 1 As shown, a power grid energy management method based on a double-depth expectation Q-learning network algorithm includes the following steps:

[0152] S1. Based on the Bayesian neural network, model the uncertainty of photovoltaic output at the prediction point and obtain the probability distribution of photovoltaic output;

[0153] S2. Input the probability distribution of photovoltaic output into the power grid energy management model based on the double-deep expectation Q-learning network algorithm to obtain the corresponding photovoltaic power generation output strategy;

[0154] S3. Operate each photovoltaic output device according to the photovoltaic power generation output strategy.

[0155] The further optimization scheme is that the establishment process of the power grid energy management model based on the double-deep expectation Q-learning network algorithm is:

[0156] T1. Only consider the energy storage system as a controllable resour...

Embodiment 2

[0183] This embodiment provides a power grid energy management system based on a dual-depth expected Q-learning network algorithm, including:

[0184] The probability distribution acquisition device models the uncertainty of photovoltaic output at the prediction point based on the Bayesian neural network and obtains the probability distribution of photovoltaic output;

[0185] The first modeling device only considers the energy storage system as a controllable resource, takes the lowest daily operating cost as the objective function and satisfies the operating constraints of the microgrid, and establishes a power grid energy management model;

[0186] The power grid energy management model of the second modeling device is modeled as a Markov decision process;

[0187] The solution device considers the random process of state transition, and proposes a double-depth expected Q-learning network algorithm by modifying the iterative rule of Q value on the basis of the traditional m...

Embodiment 3

[0191] The practical application of the present invention is explained by taking the photovoltaic output of a small industrial park from May to December of a certain year and the total load of the park as the basic data.

[0192] Assuming that the photovoltaic output and load power of the industrial park are as attached Figure 5 And attached Image 6 , 7 As shown, other parameters are listed in Table 1.

[0193] Table 1 Energy storage system parameters

[0194]

[0195] After many attempts, the sample storage capacity of the experience playback mechanism in the DDEQN algorithm is set to 4800, and the sampling size of each small batch is 600; the initial exploration rate is 0.1, the final exploration rate is 0.001, and the number of exploration steps is 24000; the learning rate is set to 0.001; update the target Q network parameters every 10 times of training.

[0196] Use the Python language and call the PyTorch package to write the Bayesian neural network photovoltaic...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a power grid energy management method and system based on a double-depth expectation Q-learning network algorithm. The method comprises the following steps: firstly, modeling photovoltaic output uncertainty of a prediction point based on a Bayesian neural network, and obtaining probability distribution of photovoltaic output; inputting the probability distribution of the photovoltaic output into a power grid energy management model based on a double-depth expectation Qlearning network algorithm to obtain a corresponding photovoltaic power generation output strategy; and enabling the system to operate each photovoltaic output device for application according to the photovoltaic power generation output strategy; according to the invention, a microgrid economic dispatching problem is simulated into a Markov decision process, an objective function and constraint conditions are mapped into a reward and punishment function for reinforcement learning, and an optimal decision is obtained by using learning and environment interaction capabilities of the reward and punishment function; the Bayesian neural network is used to model the uncertainty of the photovoltaic power generation output in the learning environment, and the state random transfer is properly considered in the Markov decision process, and therefore the convergence rate of the algorithm is significantly improved.

Description

technical field [0001] The invention relates to the technical field of power grid energy management systems, in particular to a power grid energy management method and system based on deep expectation Q-learning. Background technique [0002] With the development of renewable energy power generation technology, the penetration rate of distributed power sources such as photovoltaics in the power system continues to increase, which brings problems and even challenges to the safety and economic operation of the power system. Affected by surrounding environmental factors such as climate, the uncertainty and time-varying nature of distributed power generation such as photovoltaics have brought difficulties to the formulation of dispatch plans. How to properly model and efficiently solve the uncertainty of photovoltaic output is an important issue worthy of research. [0003] In terms of uncertainty modeling, the commonly used methods mainly include stochastic model, fuzzy model,...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06Q50/06G06N3/04G06N3/08G06N7/00H02J3/00H02J3/38
CPCG06Q50/06G06N3/08H02J3/004H02J3/381H02J2203/20H02J2300/24G06N3/047G06N7/01G06N3/045Y04S10/50Y02E10/56
Inventor 陈振韩晓言丁理杰魏巍
Owner STATE GRID SICHUAN ELECTRIC POWER CORP ELECTRIC POWER RES INST
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com