Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Decision optimization method for energy storage in transaction market based on double-Q learning algorithm

A learning algorithm and optimization method technology, applied in the engineering field, can solve the problems of overestimation of Q-learning algorithm, unstable algorithm performance, single source of decision-making information, etc., to achieve the effect of stable strategy, reduction of overestimation problem, and high income

Inactive Publication Date: 2019-12-20
NANCHANG UNIV
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The existing energy storage decision-making methods based on reinforcement learning have the following defects: only decision-making is made on electricity price information, and the source of decision-making information is single; the Q-learning algorithm has overestimation problems, and the performance of the algorithm is unstable

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Decision optimization method for energy storage in transaction market based on double-Q learning algorithm
  • Decision optimization method for energy storage in transaction market based on double-Q learning algorithm
  • Decision optimization method for energy storage in transaction market based on double-Q learning algorithm

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] The specific implementation will be described below in conjunction with the working principle of the accompanying drawings.

[0030] The double-Q learning algorithm-based energy storage decision-making optimization method in the trading market proposed by the present invention uses the double-Q learning algorithm to make decisions in the electricity and carbon market transactions in a certain place to maximize the cumulative reward. The method flow chart is attached figure 1 As shown, it specifically includes the following steps:

[0031] Step 1: Establish a mathematical model for decision-making of energy storage in the electricity market and carbon market;

[0032] Step 2: Describe the energy storage operation as a Markov decision process;

[0033] Step 3: Use the real historical carbon price and electricity price data of a trading market in a certain place, and use the Double-Q learning algorithm to iteratively train the two data sets to obtain the trained Q table; ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a decision optimization method for energy storage in a transaction market based on a double-Q learning algorithm. The method comprises the following steps: establishing a mathematical model of energy storage in-transaction market decision; describing the energy storage operation as a Markov decision process; performing iterative training on the two data sets by adopting real historical market transaction price data and applying the Double-Q learning algorithm to obtain a trained Q table; and enabling the energy storage to execute an action of maximizing the decision target in the trained Q table to obtain an accumulated reward under the joint arbitrage. According to the Double-Q learning algorithm of the invention, two functions are adopted to iteratively update theQ table, and therefore, the influence of the overestimation problem of the Q-learning algorithm can be decreased, and a designed benefit deduction strategy is more stable, and therefore, energy storage long-term benefit deduction income can be higher; the profit source is not limited to the electricity market, and the carbon market is added, so that the profit income is remarkably increased.

Description

technical field [0001] The invention belongs to the technical field of engineering. Background technique [0002] With the increasing penetration of renewable resources, it is important to efficiently achieve this balance given the high uncertainty of wind and solar energy. The energy storage system can continuously absorb energy and release energy in a timely manner to meet the large demand of users for electricity, relieve the overload of electricity on the power grid, optimize the configuration of the power grid system, maintain the complete and stable operation of the power grid, and meet the needs of different users for power. demand, is a complement to variable renewable energy, and its economic viability is gaining increasing attention. One of the most frequently discussed sources of income for energy storage is real-time price arbitrage, that is, energy storage uses the price difference in real-time electricity market prices to charge when electricity prices are low...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06Q10/04G06Q10/06G06Q30/02G06Q50/06G06N20/00
CPCG06N20/00G06Q10/04G06Q10/0637G06Q30/0206G06Q30/0226G06Q30/0283G06Q50/06
Inventor 余运俊蔡振奋
Owner NANCHANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com