Unlock instant, AI-driven research and patent intelligence for your innovation.

Single robot path planning method based on Q-Learning algorithm

A path planning and robotics technology, applied in computer parts, instruments, calculations, etc., to solve problems such as affecting the effect of path planning, insufficient exploration completeness, and missing optimal solutions.

Active Publication Date: 2019-10-25
CHONGQING UNIV OF TECH
View PDF8 Cites 15 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The cost of this approach is: the learning system will miss the optimal solution due to insufficient exploration of the environment (the value of the exploration factor ε is too small), and can only converge to a suboptimal solution, and may even converge to a Ordinary solution, this defect will affect the effect of path planning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Single robot path planning method based on Q-Learning algorithm
  • Single robot path planning method based on Q-Learning algorithm
  • Single robot path planning method based on Q-Learning algorithm

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0102] Such as figure 1 Shown: A single-robot path planning method based on the Q-Learning algorithm, including the following steps:

[0103] S1: Initialize the action set A, the state set S, the maximum number of iterations n, the maximum number of exploration steps m, the minimum number of paths MinPathNum, the maximum number of successful pathfinding MaxSuccessNum, the exploration factor ε, the single update step size of the exploration factor eSize, and the change of the exploration factor Period eCycle, maximum counting threshold h, start update time B(s, a), finish update time, action value function Q(s, a), state-action access times C(s, a), reward function storage U( s, a), the number of successful pathfinding SuccessNum, the number of successful paths PathNum, the PathList of the successful path, the successful path storage table List, the number of iterations i and the current time t.

[0104] S2: Determine whether the number of iterations i is greater than the maxi...

Embodiment 2

[0129] This embodiment discloses a simulation experiment of path planning for a single robot.

[0130] 1. Description of the simulation experiment

[0131] 1) During the simulation experiment, the software platform uses Windows 10 operating system, the CPU uses Inter Core I5-8400, and the size of the running memory is 16GB. The path planning algorithm of the single robot system will use Python language and TensorFlow deep learning tool to complete the simulation experiment, and the multi-robot path planning algorithm will be written on the matlab2016a simulation software using matlab language.

[0132] 2) This paper will use the grid method to describe the environment, divide the working space of the robot system into small grids one by one, and each small grid can represent a state of the robot system. In the map, the white grid indicates the safe area, and the black grid indicates the existence of obstacles.

[0133] The target state and obstacles in the environment are st...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of road robot path planning, in particular to a single robot path planning method based on a Q-Learning algorithm. The method comprises the following steps: initializing parameters of the algorithm; selecting an action instruction, and calculating and generating an operation state parameter and a reward function according to the action instruction; if the running state parameter is equal to the termination state parameter and is equal to the target state parameter, storing the successful path into a successful path storage table; otherwise, when theupdating starting moment is smaller than or equal to the current moment and the access frequency of the state-action pair is equal to the maximum counting threshold value, updating the action value function, and storing the running state parameters in the successful path; repeating the steps until the maximum number of iterations is reached; and repeating action instruction selection and state parameter generation according to the action value function to obtain an optimal path of the single robot. According to the method, the updating learning speed and the path planning effect of the learning system can be better improved when the Q-Learning algorithm is used for single-robot path planning.

Description

technical field [0001] The invention relates to the technical field of robot path planning, in particular to a single robot path planning method based on a Q-Learning algorithm. Background technique [0002] Mobile robots have a wide range of applications, such as home, agriculture, industry, military and other fields that have mobile robots. The three cores in the research field of controlling robot movement are robot positioning, task assignment and path planning technology. Among them, path planning is the primary condition for the mobile robot to reach the task goal and complete the task content. For example: household service cleaning robots need reasonable path planning for the indoor environment to complete cleaning tasks; agricultural picking robots need path planning to walk between crops to complete picking tasks; industrial robots also need path planning to work in shared workspaces Complete the given task. [0003] Single-robot systems are widely used in house...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62G06Q10/04G05D1/02
CPCG06Q10/047G05D1/0221G06F18/29G06F18/214
Inventor 李波易洁梁宏斌
Owner CHONGQING UNIV OF TECH