Robot reinforced learning initialization method based on neural network

An initialization method and neural network technology, applied in neural learning methods, biological neural network models, etc., can solve problems such as unstable algorithms and inability to objectively reflect the state of the robot's environment, so as to speed up convergence, improve learning ability, and improve learning efficiency effect

Inactive Publication Date: 2012-04-04
SHANDONG UNIV
View PDF4 Cites 78 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The fuzzy rules established by this method are all artificially set according to the environmental information, w

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Robot reinforced learning initialization method based on neural network
  • Robot reinforced learning initialization method based on neural network
  • Robot reinforced learning initialization method based on neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0042] The invention initializes the reinforcement learning of the robot based on the neural network. The neural network and the robot workspace have the same topological structure. When the neural network reaches the equilibrium state, the output value of the neuron represents the maximum cumulative return of the corresponding state, and the immediate return of the current state and the The maximum discounted cumulative reward of the successor state obtains the initial value of the Q function. The prior knowledge can be integrated into the learning system through the Q value initialization, and the learning of the initial stage of the robot can be optimized, so as to provide a better learning basis for the robot; specifically, the following steps are included:

[0043] 1 Neural Network Model

[0044] The neural network has the same topology as the robot workspace, and each neuron corresponds to a discrete state of the robot workspace. All neurons are only connected to neuron...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a robot reinforced learning initialization method based on a neural network. The neural network has the same topological structure as a robot working space, and each neuron corresponds to a discrete state of a state space. The method comprises the following steps of: evolving the neural network according to the known partial environmental information till reaching a balance state, wherein at the moment, the output value of each neuron represents maximum cumulative return acquired when the corresponding state follows the optimal strategy; defining the initial value of a Q function as the sum of the immediate return of the current state and the maximum converted cumulative return acquired when the subsequent state follows the optimal strategy; and the mapping the known environmental information into the initial value of the Q function by the neural network. Therefore, the prior knowledge is fused into a robot learning system, and the learning capacity of the robot at the initial stage of reinforced learning is improved; and compared with the conventional Q learning algorithm, the method has the advantages of effectively improving the learning efficiency of the initial stage and increasing the algorithm convergence speed.

Description

technical field [0001] The invention relates to a method for integrating prior knowledge into a learning system of a mobile robot and for initializing a Q value in the robot reinforcement learning process, and belongs to the technical field of machine learning. Background technique [0002] With the continuous expansion of the application field of robots, the tasks faced by robots are becoming more and more complex. Although researchers can pre-program the repetitive behaviors that robots may perform in many cases, the behavior design changes to achieve the overall desired behavior It is becoming more and more difficult for designers to make reasonable predictions about all the behaviors of robots in advance. Therefore, an autonomous robot capable of perceiving the environment must be able to learn new behaviors online by interacting with the environment, so that the robot can choose the optimal action to achieve the goal according to a specific task. [0003] Reinforcement...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/08
Inventor 李贻斌宋勇李彩虹李彬荣学文
Owner SHANDONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap