Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Preprocess method of partially observable Markov decision process based on points

A preprocessing and decision-making technology, applied in the field of acceleration of approximate algorithms, to achieve accelerated convergence, fast optimal strategy, and overcome the effects of high computational complexity

Inactive Publication Date: 2009-04-01
NANJING UNIV
View PDF0 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In conclusion, the high computational complexity has always hindered the use of POMDP for practical problems

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Preprocess method of partially observable Markov decision process based on points
  • Preprocess method of partially observable Markov decision process based on points
  • Preprocess method of partially observable Markov decision process based on points

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0024] Step 0 is the initial state of the present invention, it imports experimental data from data file;

[0025] Step 1 samples point sets through random interactions with the environment, where actions and observations are randomly obtained according to the probability of the experimental data. When the agent performs action a and obtains observation z, the belief state will be updated from b to b' according to the following formula:

[0026] b ′ ( s ′ ) = b a z ( s ′ ) = Pr ( s ′ | b , a , z ) = ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a pre-processing method of a point-based partially observable Markov decision process. The method comprises the following steps: 1. pre-processing before iteration, which comprises: a. point set is sampled by the random interaction with the environment; b. reward function of the sampling point is computed and stored; c. pseudo inheritance point is computed and stored; and d. ending; 2. pre-processing of each step of iteration, which comprises: e. a basis vector is computed and stored; and f. ending; and 3. single-point and single-step iteration , which comprises: g. a reward value table and a candidate vector table of each sampling point are computed; h. optimal action is computed and the basis vector is obtained; i. the basis vector is corrected by an error term; and j. ending. The pre-processing method of the point-based partially observable Markov decision process pre-processes each sampled belief point, provides conception of the basis vector, avoids a mass of repeated and meaningless computations, and accelerates the algorithmic speed by 2-4 times.

Description

technical field [0001] The invention relates to a method for solving a sequential decision-making model, in particular to an acceleration method for an approximate algorithm of a partially observable Markov decision-making process. Background technique [0002] In traditional multi-agent system policy problems, agents often act in completely observable environments, which makes many techniques unsuitable for practical application scenarios. Partially Observable Markov Decision Processes (POMDP) ​​provide a rich framework for sequential decision-making problems in uncertain environments. In POMDP, the state of the system and the influence of decision-making actions are not Definitely, only the observation of the hidden state can be obtained, which satisfies a certain conditional probability with the state, [0003] Since POMDP was proposed, it has received extensive attention in the field of artificial intelligence and control research, and many precise algorithms have been ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06N7/00
Inventor 王崇骏卞爱华吴骏赵志宏
Owner NANJING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com